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1. Introduction 


With the rise of global environmental issues, international 
protocols are often required to provide solutions. For instance, 
the Montreal Protocol on substances that deplete the ozone layer 
was implemented in 1987 and the Kyoto Protocol to mitigate 
climate change came into force in 2005. When protocols define 
objectives like the reduction of halogenated hydrocarbon emis- 
sions (Montreal Protocol) or greenhouse gas emissions (Kyoto 
Protocol), different measures may be implemented to achieve 
these objectives. In the case of climate change mitigation, many 
environmental policies have been proposed to substitute fossil 
fuels with renewable sources of energy. Among them, the produc- 
tion of fuels from biomass for transport is quite popular since the 
use of so-called biofuels may double over the 2010-2030 period 
[1]. However, several studies have recently highlighted an 
unwanted indirect effect of biofuel policies: the competition with 
agriculture for land use may lead to significant GHG emissions 
from carbon stock, canceling the benefits of the oil-based fuel 
replacement with biofuels over the short term [2-4]. This high- 
lights the necessity of assessing these environmental policies from 
a global perspective in order to forecast unwanted effects before 
they occur. Another point regarding these studies is the uncer- 
tainty on final results. The assumptions made in these studies and 
the models used are sources of significant uncertainty, making the 
final result highly uncertain. 

Among the tools used to study environmental impacts, life 
cycle assessment (LCA), as defined by ISO standards (ISO 14040 
and 14044), is a holistic method that makes it possible to compute 
the potential environmental impacts of a product or service based 
on several environmental impact categories. The impacts are 
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related to resource consumption or pollutant emissions occurring 
at each life cycle stage of a product or service. Recently, a new LCA 
approach, the macro LCA (M-LCA) was developed to extend the 
LCA methodology adapted to the product or service assessment up 
to policy assessment [5]. The principle of M-LCA is to use the GTAP 
model [6], an economic general equilibrium model (GEM), to 
compute the economic consequences of the life cycles changes 
planned according to a policy and then use LCA methodology to 
convert these economic consequences into environmental 
impacts. In Dandres et al. [7], M-LCA was used to evaluate two 
European Union (EU) energy policies (a bioenergy policy and a 
business as usual energy policy) over the 2005-2025 period. As in 
studies conducted by Fargione et al. [2], Searching et al. [3] and US 
EPA [4], the uncertainty related to final results is expected to be 
significant, making the identification of the policy causing the less 
environmental impacts difficult. Indeed, uncertainty is known 
to be potentially high in economic GEM [8] and LCA [9-11]. 
The investigation of uncertainty in M-LCA is therefore justified. 
For this purpose, the results of the existing M-LCA case study [7] 
conducted to assess the environmental impacts of a EU bioenergy 
policy implemented in 2005-2025 has been used. 


2. Method 


The M-LCA method (summarized in Fig. 1) uses both the GTAP 
model and LCA methodology to assess environmental impacts due 
to significant changes occurring in one or several life cycles of one 
or a group of products or services. The process relies on the GTAP 
model to compute the economic consequences of a change 
occurring in one or several life cycles and the LCA methodology 
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Fig. 1. Overview of the macro life cycle assessment (M-LCA) method. Legend: Two arrows go from the PRIMES scenario box: one arrow goes to the GTAP model in order to 
model the indirect economic consequences of the EU energy policies (and then the indirect environmental impact) while the other goes directly to the LCA database box in 
order to model the direct environmental impacts of the life cycle of the EU energy sector. 


54 T. Dandres et al. / Renewable and Sustainable Energy Reviews 36 (2014) 52-61 


to calculate the potential environmental impacts caused by the 
variations in production of each economic sector modeled by GTAP 
(i.e. the economic consequences). The results of an M-LCA compar- 
ing the environmental impacts of two EU energy policies over 
2005-2025 [5,7] is used in this paper to illustrate the method 
proposed to manage uncertainty. 

The next paragraphs describe each step of the M-LCA and the 
method used in this paper to address the different sources of 
uncertainty. 


2.1. PRIMES scenarios 


The EU energy policies studied in Dandres et al. [7] were adapted 
from scenarios developed by Mantzos et al. [12] who used the PRIMES 
partial equilibrium economic model [13] to simulate the European 
energy sector. For the purpose of the uncertainty analysis presented 
here, PRIMES scenarios are considered to be fixed parameters (i.e. no 
uncertainty is attributed to these scenarios and therefore they are not 
included in the sensitivity or uncertainty analyses). The justification 
behind this is that the objective of this paper is to study the 
uncertainty affecting the comparison of two specific EU energy 
policies rather than to study the global uncertainty related to the 
future development of the EU energy sector. Thus the two EU energy 
policies are certain and can be considered as what-if scenarios as 
defined by Borjeson et al. [14]. These two scenarios are defined in 
Dandres et al. [7] and summarized here: one assumes a significant 
increase in the use of biomass to generate energy (the bioenergy 
scenario) and the other forecasts a business as usual development of 
the EU energy sector (the baseline scenario). 


2.2. Economic evolution 


Implementing expected economic changes in prospective GTAP 
studies is commonplace [15]. In Dandres et al. [7], these changes 
were taken into account for the following macroeconomic vari- 
ables: population, labor force (skilled and unskilled), capital and 
gross domestic product. While the data used for the 2005-2010 
period are based on real monitoring, the data used for the 2010- 
2025 period are prospective estimations (see Dandres et al. [7] for 
details). Consequently, the uncertainty of the 2010-2025 data is 
expected to be greater than the uncertainty of 2005-2010 data. 
However, no information on the uncertainty of any of these data 
was found. Therefore, sensitivity analyses were conducted to take 
the uncertainty of economic evolution into account (see Section 
2.4.2 for details). 


2.3. Technological evolution 


Owing to the long temporal horizon of the study (two decades), 
technological innovation was taken into account on two levels: for 
the EU energy sector in the computation of direct environmental 
impacts and for the whole economy in the calculation of the 
indirect environmental impacts. Data collected to model the EU 
energy sector are quite detailed and provide information for each 
technology used to generate heat and electricity in Europe 
between 2005 and 2025. These data were used to modify the life 
cycle inventory in terms of the natural resource consumption and 
pollutant emissions of each EU energy technology. At the whole 
economy level, far more generic data are used to model the 
technological evolution of each economic sector of the GTAP 
model. The uncertainty of the latter data is greater than that for 
the EU energy sector level because: 


è Detailed data for each of the 6441 regional economic sectors 
of the GTAP model were not found (57 economic sectors times 
113 regions in GTAP7 database); 


@ Each GTAP economic sector includes many different technolo- 
gies, and technological innovation was therefore modeled as an 
average of the technologies included in each economic sector; 

© The technological innovation implemented in GTAP for a 
specific economic sector allows the model to reduce the 
amount of commodities needed by the sector. Consequently, 
the life cycle inventory of emitted and extracted substances 
was not modified to prevent a double counting of the reduction 
in natural resource consumption. However, the approach did 
not make it possible to model a reduction in the emissions 
specific to each technology. 


2.4. GTAP model 


The GTAP model [6] is an economic model that represents the 
global economy through 57 economic sectors and 113 regions. It 
models economic changes (via numerous economic variables) in 
response to a perturbation defined by the user. In Dandres et al. 
[7], the perturbation impacts the EU heat and electricity sectors 
and macroeconomic and technological variables. The GTAP results 
used to compute the indirect environmental impacts are the 
production variations of each economic sector in each region of 
the world (with the exception of the EU energy sector which is 
planned in the EU policies and used as input in the GTAP model). 


2.4.1. Sources of uncertainty 
There are three types of uncertainty in the GTAP results: 


1. Mathematical uncertainty due to the approximations made by 
the GTAP solver (GEMPACK) to solve the GTAP system of non- 
linear equations (thousands of equations); 

2. Model uncertainty due to the use of assumptions and equations 
that do not perfectly reflect the reality of economic and non- 
economic phenomena; and 

3. Uncertainty of the data used by the model to run simulations 
that is propagated to the GTAP results (including uncertainty in 
the variables modified by the user, also referred to as shocked 
variables). 


While mathematical uncertainty is monitored and controlled in 
GTAP model (GEMPACK monitors solutions convergence compar- 
ing solutions computed with different methods and rejects solu- 
tions when they are too different since they are interpreted to 
have a too high mathematical uncertainty, refer to Harrison and 
Pearson [16] for more details), model and data uncertainty remain 
unknown and especially hard to assess due to: 


è The difficulty in validating the model with historical data. 
Indeed, model validation with historical data is difficult to 
achieve because it would require information on a period in 
which non-economic events (natural disasters, embargos, eco- 
nomic policies, wars, etc.) did not influence the economy as 
well as data on all exogenous variables that changed during the 
considered period [6]; and 

© The high quantity of data (thousands items) that are potentially 
uncertain with an unknown uncertainty that is propagated to 
the final results of the model. 


2.4.2. Uncertainty management in GTAP 

The propagation of data uncertainty can be evaluated using at 
least two approaches. The first relies on the functionality directly 
built into GTAP that runs sensitivity analyses based on the 
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Gaussian quadrature statistical approach [17,18]. The second con- 
sists in a scenario-based approach. 

The uncertainty functionality built directly into GTAP provides 
a mean and standard deviation for each GTAP result. To proceed, 
the user must specify the type of the distribution probability and 
the range of values for each variable to be explored by GTAP in 
the sensitivity analysis. A drawback of the approach is the time 
required to run sensitivity analyses involving multiple uncertain 
variables, which can be very long, especially if variables are 
studied independently (one at a time). Concretely, the model must 
be run XN times, where X is the number of values taken by each 
variable and N is the number of variable included in the sensitivity 
analysis (also, N becomes high when all sectors and regions are 
involved in a variable). The temporal aspect becomes especially 
critical when the GTAP model is run in recursive mode, as in 
Dandres et al. [7]. In this particular case, several simulations are 
run to dynamically cover the temporal horizon, so, ultimately, the 
sensitivity analysis would require the model to be run XNM times, 
where M is the number of recursive simulations (M=8 in Dandres 
et al. [7]). Moreover, for technical reasons, internal variables and 
shocked variables cannot be analyzed at the same time, meaning 
that separate sensitivity analyses are required to account for the 
uncertainty of all variables and shocked variables. Finally, the only 
two distributions (linear and triangular) currently available for 
GTAP sensitivity analyses to characterize variable uncertainty may 
be too restrictive to reflect the real uncertainty of parameters | 19]. 
Because of these multiple obstacles, the Gaussian quadrature 
statistical approach was rejected and the scenario-based approach 
was chosen. 

The scenario-based approach has already been used by several 
authors [20-23] and consists in developing scenarios based on 
possible values for the uncertain variables and then running model 
simulations for each developed scenario. The main advantage of 
this approach is that it is more easily applied than the statistical 


Table 1 
Description of the assumptions made in each scenario and letters used to name 
each scenario. 


Scenario letter Variable group 


Armington Economy Technology 

A = = = 
B -= — 0 
€ Z = 4 
D 0 — — 
E 0 0 
F 0 — + 
G = — 
H + — 0 
I + = + 
J = 0 = 
K — 0 0 
L = 0 + 
M 0 0 — 
N 0 0 0 
(0) 0 0 + 
P 0 — 
Q + 0 0 
R + 0 + 
S = — 
T — + 0 
U = a 4 
Vv 0 + — 
Ww 0 + 0 
X 0 + + 
y + + = 
Z + + 0 
Tl 4 + Ji 
Q 0 0 0 


approach described earlier when few computational and pro- 
gramming resources are available. Indeed, both computational 
resources and programming skills can affect the time required to 
conduct sensitivity analyses in GTAP and other GEM. While 
computational resources affect the time needed to run the GEM 
simulations, programming skills may decrease the time required 
to prepare the files used to run each simulation. However, until 
computational and programming resources are available simulta- 
neously, the time required to conduct sensitivity analyses will not 
be significantly reduced. Additionally, due to the high number of 
variables in GEM, not all variables can be included in the scenario- 
based analysis in order to set up and run the required simulations 
within a reasonable time. A disadvantage of the scenario-based 
approach is that uncertainty information provided on the model 
results is less complete than those generated by a statistical 
approach such as a Monte-Carlo analysis [24]. In particular, the 
scenario approach does not yield probability distributions for 
model results. Instead, it provides worst and best cases based on 
values attributed to certain variables. 


2.4.3. Uncertainty management in the case study 

In this study, the following GTAP variables were studied for 
uncertainty: macroeconomic variables, technological variables and 
Armington elasticities. 

Macroeconomic variables were included in the uncertainty 
analysis because Dandres et al. [7] observed that they were a 
major influence on the results of GTAP simulations. Technological 
variables have smaller impacts on the GTAP simulations, but their 
uncertainty is expected to be significant. For this reason, they were 
also included in the uncertainty analysis. Armington elasticities 
are used by GTAP to handle competition between domestic and 
foreign products and are known to be very sensitive parameters in 
GTAP studies [25-28]. Therefore Armington elasticities were also 
included in the uncertainty analysis. 

Three sets of values were considered for each variable: the first 
set was obtained by increasing the original values by 50%, the 
second set was obtained by decreasing them by 50% and the third 
set was composed of the original values. 

Given the setting of the GTAP simulations (13 regions and 20 
commodities) in Dandres et al. [7], there are 65 macroeconomic 
variables, 260 technological variables and 26 Armington elastici- 
ties for a total of 351 variables. Therefore, it was not possible to 
vary each variable independently of the others due to the high 
number of GTAP simulations that would be required (8 recursive 
simulations times 37°! combinations). Instead, groups of similar 
variables were varied. Three groups were formed: a macroeco- 
nomic group, a technological group and an Armington group. 
Then, all the variables of each group were varied together, 
independently of the variables of the other groups, leading to 
the creation of 27 uncertainty scenarios (3 groups of variables and 
3 values for each group of variables), which were run recursively 
in 216 GTAP simulations (27 scenarios times 8 recursive simula- 
tions per scenario). The 27 scenarios and the letters used to refer 
to it are identified in Table 1. In this Table, "-" symbol means the 
original values are decreased by 50%; 0 means the original values 
are used without change; "+" symbol means the original values 
are increased by 50%. 

While the uncertainty scenarios cover several cases, it is 
difficult to attribute a probability of occurrence for each of them. 
Indeed, such probabilities could be computed based on the 
probability distributions of the macroeconomic, technological 
and Armington variables, but such data are currently unavailable 
due to the impossibility of comparing the forecasts with the real 
future values of the variables. The probability of each case might 
also be assessed with a panel of experts but it would require 
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Fig. 2. Differences between the baseline and renewable North American energy 
scenarios, adapted from International Energy Agency [29]. 


significant resources and the results would still remain uncertain. 
For these reasons, a simpler approach was followed. Considering 
each situation is defined by a symmetrical change ( + 50%) in the 
macroeconomic and technological variables, it was assumed each 
case has the same probability of occurrence. The limit of this 
approach is raised in the discussion. 

Considering that the evolution of the macroeconomic and 
technological backgrounds may significantly impact the results 
of an M-LCA and that North America is a major energy consumer, 
a 28th scenario (Q in Table 1) was developed based on the 
implementation of a North American renewable electricity policy 
occurring at the same time as in the EU in order to study whether 
a North American electricity policy would create economic inter- 
actions with the EU energy policy. As in Dandres et al. [7], two 
prospective energy scenarios were designed for North America. 
Both of these scenarios are adapted from the World Energy 
Outlook 2009 [29]: the first represents a business as usual 
development of the North American energy sector while the 
second forecasts a development of renewable electricity and an 
increase in energy efficiency. Fig. 2 illustrates the difference in 
electricity sources between the North American energy scenarios. 
In order to maximize the possible interaction between wood 
biomass supplies for the EU and North America, it was assumed 
that all renewable energy development in North America would 
be achieved with wood biomass. Though unrealistic, the situation 
is appropriate for the purpose of the sensitivity analysis conducted 
as part of this project. 

The GTAP simulations were conducted by integrating the changes 
in the business as usual scenario into the baseline scenario, while the 
changes in the renewable scenario were added to the bioenergy 
scenario. The ecoinvent database used to list the life cycle emissions 
and extractions of substances was also modified in order to reflect 
the changes in the North American electricity sector in each scenario. 
Then, the direct environmental impacts of North American electricity 
generation were computed and added to the EU direct environmen- 
tal impacts for each five-year period in 2005-2025. 


2.5. International databases 


The GTAP results used in M-LCA are expressed as a percent 
variation in the regional production of each economic sector. In 
order to compute the environmental impacts based on the 
production variation of each economic sector in each region, 
several databases must be synchronized to establish: 


e The production of each regional economic sector for the GTAP7 
reference year; 

© The natural resource consumption and pollutant emissions of 
each regional economic sector. 


Table 2 
Methods to manage uncertainty in life cycle assessment. 
Method References 
Sensitivity analysis 
Monte-Carlo analysis 
Bayesian Monte-Carlo analysis 
Boundaries analysis 
Multivariate regression analysis 
Combined approaches 


Cellura et al. [38] and Steen [35] 
Huijbregts [39] 

Lo and Ma [11] 

Johnson and Willis [40] 

Wang and Li [41] 

Maurice et al. [42] and May et al. [43] 


Data were therefore collected from several international data- 
bases [30-33] linked to the GTAP7 and ecoinvent LCA databases. 
Two main sources of uncertainty were identified in the process: 


1. The uncertainty of the data collected from international 
databases; 

2. The uncertainty of the linkages between the GTAP7, interna- 
tional and LCA databases (dot line in Fig. 1). 


Unfortunately, the uncertainty of the data is not documented in 
the international database websites and is therefore unknown. 
Only the Food and Agriculture Organization of the United Nations 
provides a list of potential sources of uncertainty for its data [34]. 

Two issues generate uncertainty with regard to the links 
between databases: 


1. Each database uses different nomenclatures to name its data or 
different aggregation to compute data. Therefore, it is never certain 
if a given name refers exactly to the same reality from one database 
to the next. This is especially true when names are not exactly the 
same (e.g. Molybdenum concentrates, roasted and Molybdenum, at 
regional storage) or quite imprecise (e.g. cereals). 

2. Data from different databases are not always equivalent, since 
some data may be available in a given database but not in 
another. In such situations, proxies are sometimes available and 
may be used to fill data gaps, but it is not clear how the 
uncertainty of the proxy affects the uncertainty of the final 
results in M-LCA. More specifically, it is unclear whether it is 
best to use a quite uncertain proxy or omit it and assume zero 
for the missing data. 


The higher number of data used also constitutes an obstacles when 
managing uncertainty in M-LCA: GTAP7 has 57 aggregated economic 
sectors linked to some 5000 economic activities in international 
databases and 1000 industrial processes in the LCA database (includ- 
ing 500 separate industrial processes, since a specific industrial 
process may be used as a proxy for several economic activities). 
Therefore, it is impossible to manually assess the uncertainty of each 
data (especially when linking databases) and then implement the data 
in an uncertainty management method. There is indeed a need to 
develop a method to manage such uncertainty. Additionally, data from 
international databases should be completed with uncertainty infor- 
mation that could be easily used in the uncertainty analysis. 

For all these reasons and due to a lack of resources, the 
uncertainty related to the use of international databases was not 
included in the current uncertainty analysis. 


2.5.1. Uncertainty in LCA 
Uncertainty in LCA is widely documented: 


è Steen [35] and Huijbregts [36] identified the sources of uncer- 
tainty in LCA; 
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è Weidema & Wesnaes [37] developed a pedigree matrix char- 
acterizing the quality of the data used in the life cycle 
inventory phase. 


Various approaches can be applied to manage uncertainty in 
LCA and are listed in Table 2. 

Among these methods, Monte-Carlo analysis is the most com- 
monly used and was therefore chosen to manage LCA uncertainty in 
M-LCA [44-54]. A Monte-Carlo analysis consists in attributing a 
probability distribution of possible values to each uncertain parameter 
and then computing the LCA results using a random value for each 
parameter based on its probability distribution [24]. The calculation is 
reproduced a high number of times in order to obtain a probability 
distribution for the LCA results. 


2.5.2. Uncertainty of global environmental impacts 

In Dandres et al. [7], the inventory of the natural resource 
consumption and pollutant emissions of each industrial activity is 
based on the ecoinvent (version 2.0) database, which was slightly 
modified to reflect regional diversity in electricity generation and 
include additional processes in the food economic sector (the 
structure of the database was also modified to prevent double 
counting in the environmental assessment but it is not expected to 
affect uncertainty of the results, see Dandres et al. [5] for more 
details). SimaPro (version 7.3.2) LCA software was used to manage 
ecoinvent data and, though it can run Monte-Carlo simulations, for 
technical reasons, the Monte-Carlo analysis had to be split into 
two phases using both SimaPro and Microsoft Excel with the 
Crystal Ball plug-in (version 11). The first phase of the Monte-Carlo 
simulations was conducted with SimaPro (5000 iterations) to 
manage uncertainty in the environmental impacts of each process 
in the ecoinvent database used to model the processes of the 
GTAP7 database (some 500 different processes). These Monte- 
Carlo simulations were calibrated using the mean, standard 
deviations and probability of the ecoinvent database. In this 
Monte-Carlo analysis, the uncertainty was computed for unitary 
units (e.g. 1 metric ton, 1 mî, etc.) and provided the probability 
distribution for each process and each damage category (human 
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health, ecosystem, climate change and natural resources). Then, 
the standard deviation of each probability distribution was 
increased in order to take the uncertainty of ecoinvent data used 
in the wrong context into account. Indeed, the ecoinvent database, 
which is designed for European industries and for year 2000, is 
used in M-LCA to model industrial activities in other regions of the 
world between 2005 and 2025. Since the pedigree matrix of 
Weidema & Wesnaes [37] is implemented in ecoinvent for most 
processes and may be modified with SimaPro, it was possible to 
monitor the variations in standard deviations of ecoinvent pro- 
cesses when the quality of the data is changed from best values up 
to worst values. It was found that a 2.5 factor was the biggest 
variation in standard deviation. This factor was applied to all 
standard deviations obtained in the first phase. The factor prob- 
ably overestimated the uncertainty of each industrial process 
because the 2.5 factor corresponds to the maximum gap between 
the worst and best data quality, while each process used in M-LCA 
may not have worst quality data. However, it would have been too 
time consuming to manually specify the standard deviation of 
each ecoinvent processes used in M-LCA (around 1000 processes). 

Then, the second phase of the Monte-Carlo analysis using the 
results of the previous Monte-Carlo simulations and the variations 
in the production of each regional economic sector computed by 
GTAP for each energy policy was carried out in Microsoft Excel 
with the Crystal Ball plug-in (1000 iterations) to compute the 
probability distribution of the difference in environmental impacts 
between the baseline and bioenergy scenarios for each of the 28 
uncertainty scenarios. 


3. Results 


The impact differences between the bioenergy and baseline 
scenarios for each environmental damage category in each uncertainty 
scenario are presented in Fig. 3 (each scenario is represented by a 
different letter as identified in Table 1). The differences in the imp- 
acts are computed by removing baseline scenario impacts from the 
bioenergy scenario. Thus, a positive value means that the bioenergy 
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Fig. 3. Comparison of EU energy policy environmental impacts by damage category and the percentage of cases (based on the results of the Monte-Carlo simulations) that 


the direction of the comparison computed is valid. 
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scenario causes more environmental impacts than the baseline 
scenario, while a negative value means that it causes lower impacts. 
Fig. 3 also provides the percentage of chance (based on the results of 
the Monte-Carlo simulations) that the direction of the comparison is 
valid. For instance, in scenario I, the chances that the bioenergy 
scenario causes more impacts than the baseline scenario are 77% for 
the human health category and 85% for the ecosystems category, while 
the chances that the baseline scenario causes more impacts on climate 
change and natural resources are 97 and 78%. 

It was found in Dandres et al. [7] that the bioenergy scenario 
causes lower impacts on human health, climate and natural 
resources than the baseline scenario but more impacts on ecosys- 
tems. These results are reproduced in the “N” scenario in Fig. 3 
which is referred as the “reference scenario” later in the text. 
The uncertainty analysis conducted here completes the results 
obtained previously by Dandres et al. [7] regarding the comparison 
of the two EU energy policies. 

The uncertainty analysis highlights several facts described 
below. 


e In terms of sensitivity, the difference in impacts between EU 
energy policies does not differ from the reference scenario in 
most of the uncertainty scenarios. The impacts of the bioenergy 
scenario are significantly higher than those of the baseline 
scenario only in seven cases (D, G, A, H, E, P and V), which are 
characterized by low economic growth and or low technologi- 
cal development. 

è Each environmental indicator is affected differently by the 
variations in the uncertain variables. While, in all cases, the 
bioenergy scenario causes more ecosystem damages than the 
baseline scenario, it causes more impacts on human health, 
climate change and natural resources in respectively thirteen, 
seven and five scenarios, of all 27. Thus, the environmental 
comparison of the EU energy policies made in Dandres et al. [7] 
is greatly affected for human health, slightly affected for 


Table 3 

Number of scenarios in which an opposite result to the reference scenario N is 
observed, expressed by damage categories and variation types of uncertain 
variables. 


Human health Ecosystem Climate change Natural resources 


Technological growth 


Low 6 0 5 3 
Average 4 0 2 2 
High 3 0 0 0 
Economic growth 

Low 9 0 5 5 
Average 3 0 1 0 
High 1 0 1 0 
Armington elasticities 

Low 6 0 5 1 
Average 4 0 2 2 
High 3 0 0 2 
Total 13 0 7 5 


Legend: numbers represent the number of cases where an opposite result to the 
result of the reference scenario N is observed. For instance: in 13 of the 27 
scenarios, it is observed in Fig. 3 that the bioenergy scenario is expected to cause 
more impact on human health than the baseline scenario. At the opposite, it was 
found in the N reference scenario that the bioenergy scenario would cause less 
impact on human health than the baseline scenario. Then, the 13 scenarios can be 
described according to the values taken by the group of economic, technological 
and Armington variables. Thus, it can be seen that six of the 13 scenarios have a low 
technological growth (values of technological variables decreased by 50% in 
comparison with the N reference scenario) while four occur under a regular 
technological growth (technological variables take the same values than in the N 
reference scenario) and three scenarios have a high technological growth (values of 
technological variables increased by 50% in comparison with the N reference 
scenario). 


climate change and natural resources and unaffected for 
ecosystems. 

e Macroeconomic variables are more sensitive than technological 
and Armington variables, which are almost equally sensitive. 
Indeed, Table 3 shows that a weak economic growth situation 
leads to reversed results in the EU energy policies comparison 
in 19 scenarios (nine for human health, five for climate change 
and five for natural resources) of 25 scenarios (thirteen for 
human health, seven for climate change and five for natural 
resources), while this happens in respectively 14 and 12 
scenarios for low technological development (six for human 
health, five for climate change and three for natural resources) 
and low Armington elasticities (six for human health, five for 
climate change and one for natural resources; all corresponding 
to situations in which domestic commodities are hardly sub- 
stituted by imported commodities). Additionally, most of the 
time, significant economic growth or technological develop- 
ment leads to similar results in the comparison of the EU 
energy policies found in the reference scenario. 

è The uncertainty in the impact difference between the energy 
policies differs from one damage category to the next. While 
there is less uncertainty in the climate change and natural 
resource categories (averages of 99.4 and 95.8% of the cases), 
there is more uncertainty in the human health and ecosystems 
categories (averages of 83.1 and 80.4% of the cases), including 
low values such as 50% of the cases for scenarios Q and C. 

© The higher the absolute value of the difference in impacts between 
EU policies, the lesser the uncertainty in the policy comparison 
result. Thus, the low values of some of the percentages of cases 
observed in scenarios Q and C correspond to a situation in which 
the environmental differences between energy scenarios are 
among the smallest impact differences between the 28 uncertainty 
scenarios. In other words, it means both EU policies lead almost to 
the same impacts for scenarios Q and C. 

© The 28th scenario involving simultaneous energy policies in the EU 
and North America presents lesser environmental impacts in the 
bioenergy scenario for all categories in almost all cases (average of 
96.31% of the cases). This result is explained by the significant 
reductions in coal extraction in North America and other regions 
such as in South Africa, the Russian Federation and Australia, which 
are major coal producers, while it was observed that only South 
Africa significantly reduces its coal extraction when a bioenergy 
policy is implemented only in the EU (reference scenario N). This 
result confirms that the coal sector is a key factor in this 
environmental study. Therefore, combining a coal partial equili- 
brium model and the GTAP model in order to more precisely 
model the coal sector would certainly improve the results of this 
M-LCA. 


4. Discussion 


If each uncertainty scenario has the same chance to occur in the 
future, then the bioenergy scenario causes lower impacts than the 
baseline scenario in 52% of the cases for human health, 0% for 
ecosystems, 74% for climate change and 81% for natural resources. 
Thus, if the four damage categories are considered at the same 
time, in 77% of cases, the results of the comparison of the EU 
energy policies made in Dandres et al. [7] remain unchanged when 
uncertainty in the GTAP variables and ecoinvent data is taken into 
account. However, these percentages are valid only if the uncer- 
tainty scenarios have the same probability of occurrence. Never- 
theless, the results show that these percentages would be 
significantly affected only if the probability of the cases with low 
economic growth and/or technological development would be 
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higher than the probability of the other cases. While these factors 
may remain low on a short period it is not expected they remain 
low on the long term. Thus, the risk these percentages are 
significantly wrong is expected to be low. 

Globally, the results show that, except for the human health 
indicator, the M-LCA method is relatively unaffected by uncer- 
tainty. However, not all uncertainty sources were taken into 
account. More specifically, uncertainties in the GTAP model 
(excluding uncertainty on Armington elasticities and shocked 
variables), data from international databases, database links and 
the environmental impact assessment method are not considered. 
It must also be noted that the Monte-Carlo simulation probably 
overestimated the uncertainty of the life cycle inventory for two 
reasons: all standard deviations were increased by a factor of 2.5 
(as mentioned before) and no correlations between processes 
were made in the Monte-Carlo calculation. While, in reality, some 
uncertain data for certain processes would be the same, running 
the Monte-Carlo simulation separately for each process prevents 
the use of the same random numbers when it would have been 
required to fit with this reality. Thus, independent Monte-Carlo 
simulations may exaggerate the uncertainty of the results. 

Considering all the aspects of the M-LCA, some developments 
were determined to enhance the method. 


4.1. Additional data to fill data gaps 


More LCA data are needed to reduce the data gap in database 
links and enhance the modeling of certain GTAP economic sectors: 


è The entire tertiary sector (communications, financial services, 
insurance, business services, recreational services, public 
administration, defense, education, health, and housing); 

è Certain manufactured products in the secondary sector (machin- 
ery/equipment, including transport and electronic equipment, 
chemical products, textiles and apparel); and 

è Certain activities in the primary sector (food transformation 
and mineral extraction/transformation). 


More data from international databases are required to 
improve database links and model these regional activities: 
transport (annual totals of mass transported and distance traveled 
for each mode of transport in each country), construction (annual 
new constructions by type and country) and heat generation 
(sources and annual amounts of heat generated by country). Also, 
additional LCA data are required to take the regional variability of 
industrial processes into account (ecoinvent is based on European 
data but is still used M-LCA to model industries in other regions 
despite possible technological differences). 


4.2. Additional information about uncertainty to enhance the 
uncertainty analysis 


This is especially true for data collected from international 
databases but also for models used to compute environmental 
impacts whose uncertainty is not taken into account in LCA 
Monte-Carlo simulations [55]. 


4.3. An assessment of GTAP model uncertainty 


A comparison with historical data (or another approach if 
difficulties are too important) must be carried out in order to 
assess the uncertainty of GTAP results. This assessment should 
ideally make a distinction between the contributions of model and 
data uncertainty. 


4.4. Anew integrated method to manage uncertainty in large models 


None of the techniques for variables sensitivity analysis reviews 
[56] or the uncertainty management methods developed especially for 
large models [57-59] seem adapted to M-LCA due to the high number 
of variables and equations. Two issues must be resolved in order 
to implement an acceptable uncertainty management method for 
M-LCA. First, the number of simulation iterations must be as low as 
possible to minimize computing time. Monte-Carlo simulations there- 
fore do not constitute an adequate approach to handle uncertainty in 
large models, the exception being: 


1. The availability of significant computational resources such as 
in Elliot et al. [60], which achieved a 30 000 CPU-hours Monte- 
Carlo simulation involving 2000 CPU. 

2. The achievement of new methodological developments allow- 
ing a significant decrease in required iterations. Maybe this 
development could be attained with latin hypercube [61] and/ 
or active learning [62] sampling. 


A suitable approach for M-LCA could be based on aforemen- 
tioned methods or use either the Gaussian quadrature approach 
(already implemented in GTAP) or the extended Fourier amplitude 
sensitivity testing, which is perceived as the most promising 
approach by Ravalico et al. [63] to conduct sensitivity analysis in 
environmental models used in decision-making. The second issue 
is uncertainty source management in an integrated approach. In 
the uncertainty analysis, a lot of time was spent preparing the files 
needed for each simulation, extracting and managing the inter- 
mediate data and compiling the results of the uncertainty analysis. 
Such operations could certainly be fully and efficiently automated 
with the use of a computer program that could handle the 
softwares used in M-LCA and manage all intermediate tasks. 


4.5. Further knowledge of the uncertainty related to the future 


Because M-LCA is a method to study the future consequences of 
decisions, it uses forecasts and makes assumptions about the 
future. It has been shown that macroeconomic variables play an 
important role in M-LCA simulations, so the uncertainty related to 
the future must be implemented in uncertainty management. 
However, the future is uncertain by definition, and several pro- 
spective scenarios are often used to manage uncertainty due to the 
future in LCA studies and economic modeling [14,22,55,64-72]. 
Ideally, the probability of occurrence of each prospective scenario 
should be assessed and implemented in the uncertainty approach. 
However, the uncertainty related to the future may remain 
unknown despite effort to assess it. Another type of approach 
may therefore be necessary to identify the best solution in the 
worst situation as outlined in Hites [73] rather than compute the 
uncertainty of the results. 


5. Conclusion 


In this paper, the issue of uncertainty management in large models 
is reviewed and discussed in the context of macro life cycle assessment 
(M-LCA) developed in Dandres et al. [5,7]. In the case study, the M-LCA 
results were shown to be relatively robust under the uncertainty 
conditions. The comparison of the two European Union (EU) energy 
policies showed that the EU bioenergy policy causes less environ- 
mental impacts in 77% of cases than the business as usual policy 
(percentage based on the average of all environmental indicators and 
an equal probability of occurrence of each uncertainty scenario). 
However, it is important to note that not all sources of uncertainty 
were taken into account when generating this result. A review of 
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uncertainty sources in M-LCA highlights the need for additional data, 
data of higher quality and better uncertainty management with regard 
to the computable general equilibrium model, impact assessment 
methods in life cycle analysis and prospective studies. Also, it appears 
that no method is currently able to manage uncertainty in large 
models without involving significant computational resources. There- 
fore, there is a need to develop an integrated approach that would 
manage the various sources of uncertainty when sequentially combin- 
ing models. 
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